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Abstract 

The concept of effective complexity of an object as the minimal de- 
scription length of its regularities has been initiated by Gell-Mann and 
Lloyd. The regularities are modeled by means of ensembles, that is prob- 
ability distributions on finite binary strings. In our previous paper '1 we 
propose a definition of effective complexity in precise terms of algorithmic 
information theory. Here we investigate the effective complexity of binary 
strings generated by stationary, in general not computable, processes. We 
show that under not too strong conditions long typical process realizations 
are effectively simple. Our results become most transparent in the con- 
text of coarse effective complexity which is a modification of the original 
notion of effective complexity that uses less parameters in its definition. 
A similar modification of the related concept of sophistication has been 
suggested by Antunes and Fortnow. 

Keywords: Effective Complexity; Kolmogorov Complexity; Shannon Entropy; 
Computable Stationary Processes; Coarse Effective Complexity 



1 Introduction 



The concept of effective complexity has been initiated by GeU-Mann and Lloyd 
in [5] , see also [3] . The main motivation was to define a complexity measure that 
distinguishes between regular and random aspects of a given object typically 
encoded as a binary string. This is in contrast to Kolmogorov complexity which 
is not sensitive to the source of incompressibility and in this sense fails to capture 
what is meant by complexity in the common language. 

The main idea underlying the concept has been considered at different places 
in the literature, see [4], [5], [6], [7], [9], [10]. It may be summarized as follows. 
One considers programs computing a given binary string as consisting of two 
parts: the implementation of an algorithm and a valid input for that algorithm. 
Then the corresponding measures of complexity refer to the algorithm part. 

In [2] the algorithm part has been motivated as a description of a physical 
theory represented by a probability distribution on finite binary strings while 
the second part has been used to distinguish one among all possible objects 
contained in the (typical) support of the distribution. Effective complexity is 
equal to the length of the algorithm/theory part which is minimized over the 
set of programs that compute the string and that are almost minimal, i.e. their 
length is close to the Kolmogorov complexity of the string. 

In [T] we have proposed a definition of effective complexity in precise terms of 
algorithmic information theory. Our formalization allows to include the con- 
cept into the context of algorithmic statistics, which also deals with two-part 
codings of binary strings, cf. [5]. Instances of corresponding measures of com- 
plexity are Kolmogorov minimal sufficient statistics and sophistication, cf. [5] 
and [lOj . Roughly speaking, while Kolmogorov minimal sufficient statistics of a 
binary string x is the minimal algorithmic statistic of x from the model class 
of finite sets, and sophistication refers to the model class of total programs, 
effective complexity mainly coincides with the length of algorithmic statistics 
of X minimized over computable probability distributions. For a more detailed 
presentation of the correspondence relations between the different complexity 
measures we refer to our previous paper '1 . 

More precisely, the minimization domain of effective complexity consists of com- 
putable probability distributions with total information which is approximately 
equal to the Kolmogorov complexity of the string: the tolerance level being spec- 
ified by a parameter A. Total information has been defined by Gell-Mann and 
Lloyd in [3] and [2] as the sum of Kolmogorov complexity and Shannon entropy 
of a given computable ensemble. It is worth mentioning that it is equivalent to 
the concept of physical entropy introduced by Zurek for large physical systems 
such as thermodynamic engines, [8j. 

Restricting the minimization domain of effective complexity by intersecting with 
subsets corresponding to pre-knowledge about the object, which is subjective to 
the observer, one ends up with a version of effective complexity with constraints. 
As far as we know, there is no literature other than the papers by Gell-Mann 
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and Lloyd, cf. [2], where the idea to incorporate subjective pre- knowledge into 
the measure of complexity has been considered explicitly. 

Compared to the effective complexity without constraints, which we will refer 
to as plain effective complexity or simply effective complexity, this gives a larger 
value and is the reason why Gell-Mann and Lloyd suggest to use the constrained 
version instead of the plain one: "If we impose no other conditions, every entity 
would come out simple!", see [2], page 392. This statement has to be contrasted 
with the fact that there exist strings with large plain effective complexity, cf. 
Theorem 13 of our previous work [1^. See also corresponding results in the con- 
text of algorithmic statistics and sophistication. Theorem 2.2 in 5 and Theorem 
6.5 in [5], respectively. Hence, the above conviction can be substantiated only in 
a weaker version refering to some typical behaviour. In the present contribution 
we find a framework to this end in the form of almost sure statements in terms 
of probability theory. 

In more detail, we investigate discrete-time stochastic processes with binary 
state space in the context of effective complexity as it has been defined in [1] . 
In addition to proving that typical strings are simple with respect to the plain 
effective complexity, our results also allow a deeper understanding of the depen- 
dence of effective complexity on the parameter A. Recall that this parameter 
determines the minimization domain consisting of computable ensembles with 
a total information that is A-close to the Kolmogorov complexity of the string. 
A corresponding parameter also appears in the context of sophistication and, 
more generally, algorithmic sufficient statistics, cf. [TU], [S]. Conceptually, it also 
corresponds to the significance level of Bennett's logical depth defined in [TT| . 
The relation between effective complexity and logical depth has been elaborated 
inn. 

In [To] Antunes and Fortnow suggested a modification of sophistication called 
coarse sophistication. In an analogous way, we introduce coarse effective com- 
plexity. It modifies the original concept of plain effective complexity by, roughly 
speaking, incorporating A into the definition as a further minimization argument 
and as a consequence becoming independent of this parameter. Our main results 
on effective complexity have direct implications on the asymptotic behaviour of 
coarse effective complexity. In particular, for an arbitrary stationary process the 
value of coarse effective complexity of a typical finite string is asymptotically 
upper bounded by any linear function of a string's length. 

After fixing notations and the mathematical framework in Section [2l we for- 
mulate and prove our main result. Theorem 13.11 in Section [3l It states that 
sufficiently long typical strings generated by a stationary process arc effectively 
simple. The proof relies on the observation that the total information of uniform 
distributions on universally typical subsets is upper bounded by a value that 
exceeds the Kolmogorov complexity of a typical string by any linear growing 
amount in the string's length. In Section |4] we introduce the concept of coarse 
effective complexity. We show that strings of moderate value of coarse effective 
complexity exist, see Theorem 14. 2 [ and derive from our main theorem an upper 
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bound on coarse effective compfexity of long typical realizations of a stationary 
process, see Theorem 14.31 Finally, Section [5] contains some conclusions and an 
outlook for further analysis of effective complexity in its constrained version. 

2 Notations and Preliminaries 

We denote by {0, 1}* the set of finite binary strings, i.e. {0, 1}* = {A} U 
y^gpjjO, 1}", where A is the empty string, while the set of doubly infinite se- 
quences (. . . , X-i,xo, xi, . . .) with Xi G {0, 1}, i G Z, is denoted by {0, 1}°°. We 
write £{x) for the length of a: G {0, 1}*. Finite blocks xj^ — {xm,Xm+i, ■ ■ ■ , Xn), 
TO < n, of X G {0, 1}°° are elements of {0, 1}* of length i{x^) — n - m + 1. 
We may identify them with cylinder sets [a;"J :— {y G {0, 1}°° : yi — Xi,m < 
i < n}. In a similar fashion strings x G {0,1}* are associated to cylinder sets 
of the form [x] := {y G {0,1}°° : yi — Xi,l < i < i{x)}. The tr-algebra on 
{0, 1}°° generated by the cylinder sets [a;"J, m, n G Z, to < n, is denoted by S. 
We write T({0, 1}°°) for the convex set of probability measures on ({0, 1}°°, S), 
which are invariant with respect to the left-shift T on {0,1}°°. The subset of 
ergodic T-invariant probability measures, i.e. the extremal points in 7~({0, 1}°°), 
is denoted by £{{0, 1}°°). 

Let P G ^({0, 1}°°). The random variables Xi, i G Z, given by the coordinate 
projections Xi{x) :— Xi,x £ {0, 1}°°, respectively, represent a stationary process 
with values in {0, 1}. Typical outcomes of such stochastic processes are the main 
focus in the present paper. The goal is to estimate their effective complexity. 
We will refer to elements of T({0, 1}°°) as stationary probability measures and 
stationary stochastic processes interchangeably. 

Adopting the setup of [T as far as possible we refer to probability distributions 
on {0, 1}* as ensembles. 

For each n G N we identify the joint distribution (alternatively called n-block 
distribution) P*^"^ of n successive outcomes (Xi, X2, ■ ■ ■ , Xn) of a stationary 
process with an ensembles E„ on {0, 1}* through the relation: E„(x) = P^'^\x) 
if l{x) = n and E„(a;) = otherwise. 

Recall the definition of prefix Kolmogorov complexity K{x) of a binary string 
X G {0,1}*: 

K{x) := min{^(p) : U{p) ^ x], 

where U is an arbitrary but fixed universal prefix computer. For details con- 
cerning the basics as well as deeper results on Kolmogorov complexity theory 
we refer to the book by Li and Vitanyi [T^. 

We call an ensemble computable if there exists a program for the universal 
computer U that, given x G {0, 1}* and to G N as inputs, computes an approx- 
imation of the probability E(a;) with accuracy of at least 2~™. 

In [I] we have introduced an extension of the notion of Kolmogorov complexity 
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to the case of computable ensembles E with computable and finite entropies 
iJ(E). Here we mean by entropy iJ(E) the Shannon entropy of the probability 
distribution E defined by — E(a;) logE(x). Note that a computable 

ensemble does not necessarily have a computable entropy, such that the corre- 
sponding requirement is a restriction, see |Ij for details. In what follows a dis- 
tinction only between general ensembles and computable ones with computable 
and finite entropies is drawn. We will refer to the latter ones as computable for 
short. 

The Kolmogorov complexity K{K) of a computable ensemble E is defined as the 
length of the shortest computer program that, given x G {0, 1}* and m e N as 
inputs, outputs both E(a;) and iJ(E) with an accuracy of at least 2^™. 

Additionally, we need to define computability of stochastic processes. The fol- 
lowing definition is a reformulation of the notion of a "computable measure" in 

m- 

A stationary process P is called computable if there exists a program p S {0, 1}* 
for a universal computer U that, given x G {0,1}* and m G N as inputs, 
computes the probability -P([a;]) up to accuracy 2~™. 

3 Effective Complexity of Stationary Processes 

The goal is to show that under not too strong conditions long typical samples of 
stationary processes are effectively simple. Before we make rigorous statements 
we need a number of definitions. The first ones we adopt from our previous 
paper [T]. 

Let (5 > 0. We say that an ensemble E is S-typical for a string x G {0, 1}*, or 
alternatively, we call x (5- typical for E, if the Shannon entropy H{E) of E is 
finite and 

-logE(x) < H{E){l+6). 

In particular, the special case of an equidistributed ensemble is (5-typical for all 
strings in the support and any S > 0. 

The total information I](E) of a computable ensemble E is defined by 

S(E) := H{E)+K{E). 
For a motivation of the two above defintions see [1]. 

Let S > and A > 0. Effective complexity £s,Aix) of a finite string 
X G {0, 1}* is defined by 

Ss.Aix) ■■= min{A'(E) : E G VsAix)}, 
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where V5,a{x) denotes the minimization domain associated to x: 

Vs.a{x) := {E : E computable ensemble, E (5-typical for x, I](E) < K{x) + A}. (1) 

We refer to elements of Vs.Aix) as effective ensembles for x. 

Using the viewpoint of which was reviewed in [1 , effective ensembles 
represent theories or explanations that are judged to be good for x. 

The more general notion of effective complexity with constraints has been 
suggested in [2] mainly to circumvent problems of plain effective complexity. 
We have discussed them shortly in the Introduction. The main idea is that the 
constraints reflect some pre-knowledge about the possible theory for x. In [T] 
we have proposed a formalization of the constrained version in the following 
manner: 

£5a{x\ C) TcAn{K{¥.) : E e VsAi^)^^^ C}, 
where C is a subset of 7'({0,1}*). Note that with C = P({0, 1}*) we have 
£sa{x) = £&a{AC), for aU x £ {0, 1}*. 

In what follows other essential concepts are that of typical and/or uni- 
versally typical subsets. 

Let P be a T-invariant probability measure on ({0,1}°°,!]). We call a 
sequence of subsets M„ G S, n G N, P-typical if 

lim P{Mn) = 1. 

n— ^oo 

We call (M„) strongly P- typical if for P-almost all x there exists an Nx G N 
such that 

X G Mn for every n > N^- 

The above notions of typicality apply naturally to sequences M„ C {0, 1}", 
n G N, if we identify subsets M„, n G N, with cylinder sets [M„] C {0,1}°°, 
respectively. 

Let A be a set of stationary processes with values in {0, 1}. We call 
Mn C {0, 1}", n G N, universally typical for A if the sequence is P-typical 
for every P G A, i.e. liiRn^oo P'^^'KMn) = 1- Wc caU the sequence universally 
strongly typical for A if it is strongly P-typical for every P G A. 

For sets A^ C £{{0, 1}°°) consisting of ergodic processes with entropy rate 
upper bounded by r > there exist universally typical subsets Tr^n C {0, 1}" 
with 

\T„r\ < 2™ (2) 

for all n G N. Moreover, there are methods to construct such sequences of 
universally typical subsets for A,.. We will apply the Lempel-Ziv algorithm in 
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the construction procedure below, see [131 [TU [T5] . The main point is that all we 
need to know about an ergodic process P is its entropy rate hp. This allows to 
prove the following theorem for stationary, in general not computable processes. 



Theorem 3.1 Let P he a stationary process, 5 > Q, e > Q and A„ = en. Then 
P is effectively simple in the sense that for P-almost every x, 

Ss.Ar. {xi ) < log n + O (bg bg n) . (3) 

Proof. 1. Assume that P is an ergodic process with entropy rate hp. We construct 
universally typical subsets T^.n Q {0, 1}", rt € N, such that for an appropriate 
choice of the parameter r — r{hp, e) the total information of the uniform distri- 
butions Er,„ on Tr^n C {0, 1}", respectively, is upper bounded by K{xi) + A„ 
of P-almost every x € {0, 1}°° and for n large enough. Hence the Kolmogorov 
complexity of Er,„, which is approximately upper bounded by logn, gives an 
estimate from above on the effective complexity £s.A„ix^) of sufficiently long 
P- typical strings x". 

First, let r > be arbitrary and define for each n G N the subset Tr^„ C {0, 1}" 
as the set consisting of all binary strings x" which are mapped by the Lempel- 
Ziv (LZ) algorithm to a code word of length £lz{xi) lower than nr. Then the 
sequence Tr^„, n e N, is universally typical for the set of ergodic processes 
with entropy rates lower than r. 

Recall the following remarkable property of the LZ algorithm: For every ergodic 
process Q with entropy rate hg it holds lim„_>oo ^£lz{x") = hq for Q-almost 
all X G {0, 1}°°. This implies that indeed the subsets T^.n as constructed above 
are typical for any ergodic Q with hq < r. 

The upper bound ([2]) on the size \Tr,n\ follows from the fact that the LZ algo- 
rithm is a faithful coder and as such satisfies the Kraft inequality: 

J2 2-'"' = |T,,„|2-"''. 

Next, we show that if r is chosen to be a positive rational number satisfying 
< r — hp < e/4 then for P-almost every x G {0, 1}°° there exists an G N 
such that 

E(E^,„) <K{x'l) + An for every n>N^, (4) 

where again Er,„ denotes the uniform distribution on the universally typical 
subset Tr^n- First note that for all ti G N 

i7(E,,„) = log|T,^„| <rn. 



1 > 

> 
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Secondly, we prove that there is a constant c G N such that for all rt G N 

K{Er^n) < K{n) + K{r) + c. (5) 

This is derived from the existence of a program p of length c which expects as 
inputs n e N, r e Q and x G {0, 1}* and outputs the value ^rj} | if a; G T^.n C 
{0, 1}" and otherwise. Thus for fixed inputs n and r it gives a description of 
the uniform distribution Er.„ on T^^n- 

Indeed, p may be constructed on the base of a program p^z implementing the 
Lempel-Ziv (LZ) algorithm on the given reference universal computer U . For 
given inputs n and r let p apply p^z as a subroutine in order to determine ele- 
ments of Tr^n- Then for fixed n G N the number iT^.n! and hence the probability 
value l/|Tr.„| of each x G Tr^n may be calculated easily. 

To specify r G Q and n G N a number of K{r) + K{n) bits is sufficient. With 
c = i{p) the estimate (HJ follows. 

Next, fix an iV G N such that K{n) + K{r) + c < for all n>N. Then 

S(IEr,«) <nr + K{n) + K{r) +c<n{r + ^)< n{h + |), (6) 

where the last inequality holds by assumption r — h < |. According to the 
theorem by Brudno, see [TB], for P-almost all x there exists an N^^e G N such 
that K{xi) > n{h — |) for all n > N^.e- It follows for A„ = en 

if«) + A„ >n(/i+|), n>iV,,,. (7) 

Relations © and (O together imply Q for P-almost all x and n > A^a; := 
max{Nx,e, N}. It follows that P-almost surely the effective complexity £s.a„ (a^i ) 
is upper bounded by the Kolmogorov complexity of Er,„ for n > N^: 

Ss,aAxi) < K{Er,n) < K{n) + K{r) + c 
< log n -t- ©(log log n). 

2. Now let P be an arbitrary stationary process. Recall that there is a unique 
ergodic decomposition of P 

P = / Qdii{Q). 

J£({0,1}°°) 

Moreover, to P-almost every x G {0, 1}°° we may associate an ergodic compo- 
nent Qx of P such that a; is a typical element of Qx- Then there exists an N^^e 
such that 

K{x1) -h A„ > n{hx -t- |), n > Nx,e, 

where denotes the entropy rate of Qx- Hence the proof for the stationary 
case reduces to the ergodic situation considered in the first part above. □ 
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4 Coarse Effective Complexity 



Our main result becomes most transparent if presented in the context of 
coarse effective complexity. This is a modification of plain effective complexity 
which incorporates the parameter A as a penalty into the original formula. 
It is inspired by a corresponding modification of sophistication, called coarse 
sophistication, which has been introduced by Antunes and Fortnow in [lOj . 

Let 6 >0. The coarse effective complexity £s{x) of a finite string x € {0, 1}* is 
defined by 

£s{x) '■— mm{K{E) + S(E) — K{x) : E is computable ensemble, E, S — typical for x}. 

The term S(E) — K{x) accounts for the exact value by which the total informa- 
tion of an ensemble E exeeds the Kolmogorov complexity of x. By definition of 
total information I](E) an equivalent expression for Ssi^) reads 

£s{x) = min{2if (E) + H{K) — K{x) : E is computable ensemble, E J — typical for x}. 

We derive the basic properties of coarse effective complexity similarily as it 
has been done in [TU] in the context of coarse sophistication. That is firstly, in 
the proposition below, we prove an upper bound on coarse effective complexity. 
Secondly, we show existence of strings, which are close to saturate this bound. 

Proposition 4.1 Let S > 0. There is a constant c such that for all x G {0, 1}* 

we have 

Ssix) < -+\ogn + c, (8) 

where n = £{x). 

Proof. Suppose that K{x) < ^-l-log n. Let E^, denote the ensemble with E(a;) = 1 
and K{y) — for y ^ x. Note that E^, is trivially 5-typical for x for any S > 
and obviously i?(E^) — 0. Moreover, there is a constant ci such that it holds 
K(Ex) ^ K{x) + ci. This implies the upper bound 

Ss{x) < 2K{Ex) + - K{x) 

< K{x) + 2ci 
n 

< -+logn + 2ci, 
where the last line holds by assumption. 

Now, suppose that K{x) > f + logn. Let E„ be the ensemble on {0, 1}* given 
by E„(y) = ^ for all y with £{y) = n and vanishing elsewhere. Then i/(E„) = n 
and there exists a constant C2, independent of n, such that if (E„) < logn + C2. 
It follows 

£s{x) < 2\ogn + 2c2+n- K{x) 
n 

< -+logn + 2c2. 
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where, again, the second line holds by assumption on K{x). Setting c := 
max{2ci,2c2} completes the proof. □ 

Theorem 4.2 Let 5 >Q. For every sufficiently large n S N there exists a string 
X £ {0, 1}" with 

£s{x) > (1 - 35)- - (2 + 3,5)logn- 21oglogn + C, (9) 
where C is a global constant. 

Proof. For x G {0, 1}* and A > denote by the minimal ensemble associated 
to £s,a{x)- Due to Lemma 22 in [1 for every e > there exists a subset of 
{0,1}* such that 

log 15^1 < HiE^)il + S)+e (10) 
KiS^) < KiEf) + c,, (11) 

where ci is a global constant. In [T] we have proven the relation 

K{x\StK{S^)) > i^fi^ - logn - 21oglogn - Aa, (12) 

1+0 

which holds for arbitrary x G {0, 1}", n G N. The term Aa is constant in 
X e {0, 1}* and monotonically increasing in A, cf. (32) in [1]. Now, let Kn ■= 
max{i^(i)| t £ {0, 1}"} and define 

k -.^ n- 5{Kn + A„ + e) + logn + 2 log log n - Aa„ - C2, 

where A„ := ^ +logn + c is the upper bound on £s{x) obtained in the previous 
proposition and C2 is a global constant from Theorem IV. 2 in [5], see also Lemma 
12 in [1]. If n is large enough then < fc < n holds, and Theorem IV. 2 in [5] 
applies: There is a string Xk £ {0, 1}" such that 

K{xk\S,K{S)) <\og\S\-n-k + C2, (13) 

for every set S 3 Xk with K{S) < k — cs, where C3 is another global constant. 
Let Ex denote the minimizing ensemble associated to coarse effective complexity 
Ssix) and A^ := K (E^) + H {E^) - K (x) such that E5 (x) = i^(E^)+A^. Further, 
define Sx '■= S^'^ . It holds the inequality 

-S{Kn + A„ + e) < -~diK{xk) + Ax + e) 
< -SiH{Ex,) + e) 

e 



< -~S\H(Ex,) + - 



l + S 



(i/(E,J(l + 5) + e) 
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where the last upper bound holds by ([TU|) . Now suppose that K{Sxk) < k — ci. 
Then 

< 



< 



< 



But the strict inequality is a contradiction to (1121) . Hence our assumption must 
be false and we instead have K{Sx,.) > k — c^. By £s{x) — K {E,x) + and using 
both (fTTj) and the bound Kn < n + 2 log n + 7, where 7 is a global constant, we 
finally obtain 

£sixk) = 
> 
> 

> 



where C := -5{j + e) - 1 - (1 + 5)c - ci - C2 - C3. □ 

Although, according to the above theorem, for arbitrary large n the exis- 
tence of strings of length n with moderate coarse effective complexity is 
ensured, the coarse effective complexity of sufficiently long prefixes of a typical 
stationary process realization becomes small. This is a direct implication of 
Theorem 13.11 

Theorem 4.3 Let P be a stationary process, 6 > and e > 0. Then for P- 
almost every x 

£s{xl) < en + logn + O (log log n). (14) 

Proof. By definiton of coarse effective complexity it holds £s{x) < A + £s,a{x), 
for all a; e {0, 1}* and A > 0. We set A„ — en. Then the conditions of Theorem 
13.11 are satisfied and applying ^ we arrive at ([T^ . □ 



log \SxJ -n + k + C2 
log \Sx, I - log n - 2 log log n 
-Aa„ - S{Kn + A„ + e) 



1+^ 

-Aa„ 
1 + .5 



log 71 — 2 log log n 
log n — 2 log log n 



fc - C3 - Ci + Axfc 

3 

n — 6(2''^ + ^ logn + 7 + c + e) — logn 
n 

-2 log log n - - - log n - c - C2 - 1 - C3 - ci 
(1 - 3(5) - - (2 + '36) log n - 2 log log n + C, 
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5 Conclusions 



In this contribution we studied the notion of plain effective complexity, which 
is assigned to a given string, within the context of an underlying stochastic 
process as model of the string generating mechanism. In [T] we have shown 
that strings which are called "non-stochastic" in the context of Kolmogorov 
minimal sufficient statistics have large value of plain effective complexity. The 
existence of such strings has been proven by Gacs, Tromp and Vitanyi in [5]. 
Here, our aim was to understand how properties of the stochastic process such 
as ergodicity and stationarity influence the effective complexity of corresponding 
typical realizations. Is it possible that the prefixes of a typical process realization 
represent a sequence of finite strings in increasing lenght n that eventually have 
a high or moderate value of effective complexity? Our main theorem refers to 
stationary and in general non computable processes. It proves that modelling 
the regularities of strings by computable ensembles with total information that 
is allowed to excess the string's Kolmogorov complexity up to a linearly growing 
amount en with an arbitrary small e > is sufficient for typically generating 
non-complex strings. 

The value en plays the role of a parameter in the concept of effective complexity. 
In order to have a notion that is independent of this parameter we introduced 
coarse effective complexity. It corresponds to coarse sophistication introduced by 
Antunes and Fortnow in [T^ and modifies effective complexity by incorporating 
the parameter as a further minimization argument. Our result on effective com- 
plexity has a direct implication on the asymptotic behaviour of coarse effective 
complexity of typical realizations of a stationary process. The main statement 
in this context demonstrates the utility of the linear parameter scaling which we 
have considered. Moreover, it allows to analyse the interplay between the com- 
plexity of a stochastic process and the complexity of its typical realizations. In 
particular, it demonstrates that, in order to have a notion of effective complexity 
that also reflects the complexity of a stochastic process, further modifications of 
plain effective complexity are necessary, for instance introduction of appropriate 
constraints. This possibility is in line with Gell-Mann and Lloyd's suggestion in 
[5] which we discussed in the Introduction. 

Finally, we point out that continuing our previous work ^1^ we have formulated 
our results for the concept of effective complexity only. However, in line with 
the general equivalence statements obtained in the literature, cf. Section V in 
[B] or Lemma 20 in [T], it should be possible to reformulate our main theorem 
in the more general context of algorithmic statistics. Indeed, our upper bound 
on effective complexity of typical process realizations is derived in terms of 
computable ensembles that are uniform distributions on finite sets (universally 
typical subsets). This demonstrates the close relation in particular to the concept 
of Kolmogorov minimal sufficient statistics which refers to the model class of 
finite sets. 
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